Utilizing Distance Metrics on Lineups to Examine What People Read From Data Plots

摘要

Graphics play a crucial role in statistical analysis and data mining. Thispaper describes metrics developed to assist the use of lineups for makinginferential statements. Lineups embed the plot of the data among a set of nullplots, and engage a human observer to select the plot that is most differentfrom the rest. If the data plot is selected it corresponds to the rejection ofa null hypothesis. Metrics are calculated in association with lineups, tomeasure the quality of the lineup, and help to understand what people see inthe data plots. The null plots represent a finite sample from a nulldistribution, and the selected sample potentially affects the ease ordifficulty of a lineup. Distance metrics are designed to describe how close thetrue data plot is to the null plots, and how close the null plots are to eachother. The distribution of the distance metrics is studied to learn how wellthis matches to what people detect in the plots, the effect of null generatingmechanism and plot choices for particular tasks. The analysis was conducted ondata that has already been collected from Amazon Turk studies conducted withlineups for studying an array of data analysis tasks.